Projects: LOOPS : 2008 Annual Report draft

This page last changed on Oct 24, 2008 by stepheneb.

Only edit this document using Wiki Markup editing.
Do NOT edit this document using WYSIWYG (RTF) Markup.

LOOPS Annual Report draft

Project Partners

Concord Consortium

Concord Consortium activities

Concord Consortium Publications and Presentations

Concord Consortium Participants

The following lists staff at CC who have spent 160 hours or more on the LOOPS project:

Bob Tinker 315 hours

Robert Tinker is the PI of the project, overseeing all aspects of the scientific and educational quality of the project.

Specifically Bob has worked on planning the project startup and early meetings including a meeting of the three collaborating teams (CC, UC Berkeley, and the Ontario Institute of Science Education) here at Concord on February 14, 2008. Bob has contributed to the design of the learning activities including a detailed outline for the Force and Motion curriculum and has provided review and oversight of SAIL technology development.

Stephen Bannasch, 510 hours

Stephen Bannasch is the Director of Technology, responsible for the underlying architecture.

Stephen participated in early LOOPS planning meetings at UC Berkeley Oct 9-11 2007 with Marcia Linn, Jim Slotta, and Michelle Spitulnik. The focus of these meetings was to bring the partners up-to-date with progress at CC on related SAIL/OTrunk technologies under development in the UDL and ITSI projects at CC and work towards statements of work for the sub-grantees. In addition Stephen visited the Martinez middle school to observe a Global Warming TELS classroom run and to interview the teacher.

Stephen participated in the SAIL technical retreat Oct 24-28 in Ontario Canada. The main LOOPS goal was to affect the future architectural direction for SAIL to better support integration of authoring, reporting of student data and scripting. This was a success as the SAIL community decided to integrate CC's OTrunk framework into SAIL.

Stephen developed the statement of work for the OISE group in Toronto and the Open Source IP agreements used among all three collaborators.

Stephen both develops and manages software development for the LOOPS project. These efforts include performance improvements for the SAIL Data Service, development of new Reporting architectures, definition and development of the LOOPS Teacher Dashboard.

In addition Stephen has been worked to improve collaboration tools used by the technical, pedagogic, and research teams in the project by reorganized the project Confluence wiki site and setting up mail-lists, an IRC channel, simpler access to example code, and new forms of distributed source code systems.

Stephen developed the core of the one-dimensional motions prototype LOOPS activities demonstrated at the August 2008 TELS Retreat and in addition developed and demonstrated the new LOOPS reporting architecture using JRuby scripting and ERB templates.

In addition Stephen has been working on architectural changes which will allow developing a LOOPS server system deployable to a school which will greatly increase the speed an reliability when running activities and reporting in the classroom.

Scott Cytacki, 307.5 hours

Scott Cytacki is the senior software engineer supervising the development of software for the project.

Scott has participated in all the planning of LOOPS software development. Scott is the Java chief architect of the SAIL/OTrunk system and as such contributes in numerous ways that improve stability, functionality, and documentation of the framework.

Scott has worked with Stephen on both generalizing the DIY portal/authoring/deployment system as well as adding LOOPS-specific reporting functionality.

Scott created the initial example of a JRuby/ERB based OTrunk report. This style of report is an easy to customize way of showing the work of multiple students. The combination of this work with the new OTrunk Overlay system provides a way for teachers to give feedback to students or customize the activity content based on data in student reports. It is this work that is being extended into the LOOPS Teacher Dashboard.

An ongoing effort of Scott's has been the integration of the JackRabbit implementation of the Java Content Repository system into the SAIL/OTrunk layered persistence architecture. The goal is to both add support for integrated OTrunk persistence of author, and teacher changes to activities as well as supporting semi real-time persistence during a classroom session.

Additionally both Scott and Stephen have been working to better integrate and support automated testing of the underlying SAIL/OTrunk frameworks as well as higher-level LOOPS activities and reports.

Scott is also supporting the team at UC Berkeley in the development of the new Project Service Layer functionality for the TELS Portal. It is this abstraction which will allow the TELS Portal to be able to run both legacy Wise projects n SAIL as well as the newer LOOPS projects.

Aaron Unger, 240 hours

Aaron Unger is a junior programmer at CC who works on OTrunk models, Reporting, and the supporting SAIL/OTrunk frameworks.

Aaron has been both adapting older models into OTrunk to improve our ability to generate useful reports from student interaction and developing new models in CC's Molecular Workbench authoring system for use in LOOPS.

Aaron also works with Stephen on performance and functional improvement of the SAIL Data Service to support both increasing load as well as new reporting needs.

Aaron also supports the UC Berkeley technical team in their adaptation of the OTrunk framework for TELS.

Paul Horwitz, 220.5 hours

Paul is a co-PI of the LOOPS project at CC and is a Senior Scientist working on curriculum design and reporting strategies.

Paul has participated in planning meetings on curriculum and assessment implementations, reviewed the California 8th grade standards for Force and Motion and adapted them for LOOPS, worked on reviewing and adapting existing research-based physical science curriculaa materials to make them useful to the LOOPS project, and completed design of LOOPS units for force and motion.

In addition Paul researched ThinkerTools curricula for potential application to LOOPS.

Ken Bell, 220.5 hours

Ken has supported the entire project team with planning and scheduling of the various project meetings and collaborations. These include: planning for Berkeley and Toronto sub-award LOOPS letters of agreement; scheduling for LOOPS Phase 1 and 2 and the Feb 14 2008 Partners kick off meeting with Berkeley and Toronto. In addition Ken coordinates the LOOPS leaders teleconference meetings, works with the Berkeley team to select LOOPS classrooms and teachers; supports classroom pilots by coordinating the delivery of probeware and computer equipment; and managed curriculum development with the teacher-consultant Jeff Schoonover and Paul Horwitz

Ken has also completed a first draft of a literature review of research on student physical science misconceptions for LOOPS.

In addition Ken developed and tested a more extensive protoype OTrunk LOOPS activity template.

Seong June Kim, 145.5 hours

Seong June Kim is a junior programmer who has been contributing to the development of new reporting systems for LOOPS.

Seong worked with the technical developers at to design, implement, and test aggregate reporting on learner data stored in the SAIL Data Service. Seong also worked with Scott on the new data persistence scheme using JackRabbit implementation of a Java Content Repository.

Paul Burney, 132 hours

Paul Burney was the webmaster at CC; he designed the LOOPS project website and prototype activity portal.

Paul developed the initial LOOPS project website and extended and documented CC's existing PHP-based SAIL/OTrunk Portal to support prototype LOOPS projects with access to the new reporting functionality exposed through the DIY portal system.

In addition Paul worked to maintain and improve the performance of the supporting web and database systems at CC to support the LOOPS project.

Sam Fentress, 75 hours

Sam Fentress is a junior programmer who is contributing to new Smart Graph, Smart Table, and Smart Model functionality for LOOPS.

Sam worked with Stephen to improve the OTrunk WYSIWYG authoring tools and Data Table capabilities for the demo of prototype LOOPS activities for the August 2008 TELS Retreat.

Sam's work on the Smart Graph includes new capabilities that can be used to extract data from graphs and display and/or highlight for the learner interesting features. In addition Sam has worked with Stephen to extend the capabilities of the scripting interface for customizing reporting into OTrunk.

Ben Greslick, 7.5 hours

Ben worked with the developer team at CC to create a bootable USB Linux server that has most of our UDL/LOOPS/java software and portal that can be deployed to schools with bandwidth issues.

Cynthia McIntyre, 3 hours

Cynthia wrote a press release and submitted it to PR Newswire. In addition she has written a news announcement for @Concord newsletter.

University of California, Berkeley

University of California, Berkeley activities

The UC Berkeley subcontract for the LOOPS project was executed in 2008. Leadership includes Marcia Linn (Director) and Kathy Benemann (Manager), Doug Kirkpatrick (Program Coordinator). Graduate students include Kevin McElhaney, Helen Zhang, Phil Daubenmeir, and Jenny Chiu. Technology staff include Hiroki Terashima, Geeff Kwan, and Tony Perritano. Staff include David Crowell and Jon Breitbart. The Berkeley Institutional Review Board has accepted UC Berkeley's LOOPS protocol. We are excited to expand on our collaboration in TELS into this new project.

UC Berkeley participants have contributed to a face-to-face meeting in Concord and regular technology and leaders meetings averaging 3 times per month. In addition, Bell and Benemann have coordinated on a weekly basis. In addition, we have contributed to the WIKI for LOOPS.

During the first year we have reviewed and pilot tested possible items for baseline assessments, recruited teachers and schools, began the process of defining LOOPS scenarios, and conducted meetings with potential LOOPS users. In addition, we have reviewed possible curriculum and technology designs and considered ways to incorporate LOOPS.

Assessment

Baseline assessments for Force and Motion and Chemical Reactions items were administered to both TELS and non-TELS students based on convenience. The two groups are not comparable as they come from schools with different Academic Performance Index scores. These are all items that have been released by TIMSS or other testing programs.

The pilot tests can be used to establish goals for the new curriculum materials. As the bar chart below shows, performance on Force and Motion items suggests a need for intervention in student learning in this topic area (Figure 1). For schools that did not study TELS, this can be seen as a baseline. As is apparent, students have limited understanding of the topics in these assessments.

Figure One: Five items administered to students in TELS schools who either did or did not study any TELS units the previous year.

The Chemical Reactions items were administered to students in similar schools. An analysis of Chemical Reactions items indicates significant advantage to students who experienced TELS instruction consistent with the impact of the unit in the past (Figure 2).

LOOPS Schools

UC Berkeley has recruited three schools to participate in the LOOPS project and one school to participate as a potential back-up school. All of these schools have participated in past TELS-related research programs and have garnered administrative and district-level support for integrating effective use of technology into science curriculum. Several formal and informal meetings have been organized since June 2008, during which we collaborated with teachers and researchers in the curriculum and technology design process.

We secured pacing guides for physical science courses. We have reviewed these guides as well as the California standards. Since this is the first year of use of the guides we envision some revision after the plan is tried out. The timing of the units and the time devoted to topics appears a bit disjointed. To make LOOPS successful we will need to align the guides with our curriculum design plan.

Our first formal meeting in July provided us with invaluable insight into teacher's successes and challenges when teaching Force and Motion topics to middle school students. Most of the teachers expressed excitement about their prior use of motion probes and indicated interest in continuing to use this technology. The teachers were somewhat confused about the goals of LOOPS and how the program would be enacted in their classrooms. In response we began the process of creating scenarios. See notes.

During our second formal meeting in August, attended also by Concord Consortium project manager Ken Bell, we were able to gather teacher perspectives into revising current TELS modules to fit LOOPS research and instructional goals, while also evaluating new technology tools. Kevin McElhaney, who has been serving as the LOOPS graduate student researcher, played an essential role during these planning meetings, providing both teachers and senior researchers avenues for designing activities and the accompanying technology components. See notes.

These meetings underscored the importance of clarifying LOOPS scenarios and developing proof of concept technologies. These materials will be discussed with teachers as they become available.

LOOPS Scenarios

We have participated in the iterative process of drafting LOOPS scenarios (see current memo, v4). These discussions are helping us identify the kinds of feedback systems that might be desirable in the classroom setting. These scenarios have been revised by UC Berkeley, Concord Consortium, and Toronto. We will soon discuss these ideas with teachers and further refine the scenarios.

LOOPS Curriculum materials

We have discussed various curriculum approaches, recently meeting with Tinker at Berkeley to refine the context of instruction and consider appropriate pacing arrangements.

University of California, Berkeley Publications and Presentations

University of California, Berkeley Participants

Marcia Linn

Marcia Linn is a co-PI of the LOOPS project and the Director of the LOOPS project at UC Berkeley.

Kathy Benemann

Doug Kirkpatrick

Kevin McElhaney

Helen Zhang

Phil Daubenmeir

Jenny Chiu

Hiroki Terashima

Geoff Kwan

Tony Perritano

David Crowell

Jon Breitbart

University of Toronto

University of Toronto activities

Continued development of the Scalable Architecture for Interactive Learning (SAIL)

Continued development of the Scalable Architecture for Interactive Learning (SAIL).

Since 2003, Professor Slotta has led an international team of researchers and technology designers in developing SAIL, which is a java-based framework for the development of interactive, interoperable learning materials and environments. SAIL has been the basis of our development activity in the NSF-funded TELS center, and is one of the primary deliverables of that effort. It has served as the basis of all technology development in subsequent projects, including LOOPS and CLEAR (Cumulative Learning using Embedded Assessment Results -- another Dr. K-12 project that came out of TELS, and relies on its core technologies). Slotta has led the SAIL technology design, focusing on the emergence of a co-design community where numerous distinct development communities can participate and collaborate in creating open source and open content tools, materials and environments.

Organized a SAIL technology architecture workshop.

Organized a SAIL technology architecture workshop. June 23, 2008. Utrecht, The Netherlands

Professor Slotta organized, with collaborator Turadg Aleahmad and Stephen Bannasch, a pre-conference workshop where members of the international community were exposed to the core LOOPS technologies and encouraged to design new applications that would connect to their own research. 15 participants gathered in Utrecht, the Netherlands, including representatives from a large European Union Framework 7 project called SCY that is interested in collaborating in technology development, as well as members of prestigious U.S. and Scandinavian labs. This workshop was a full day event, with participants first exploring SAIL and LOOPS technologies, then breaking into focus groups (one on technology architectures and repositories, and another on curriculum and assessments).

Convened a SAIL repository development workshop.

Convened a SAIL repository development workshop. August 1-5, 2008. Berkeley, California.

Building on the earlier technology workshop, Professor Slotta invited the two lead technology developers from the SCY (Science Created by You) project to participate in a hands-on development workshop held in Berkeley, California. SCY is a large collaboration project funded by the European Union's Framework 7 program (8.5 Million Euros from 2008-2013). Professor Slotta is a partner in this project, as he is eligible being from a Canadian institution. Slotta is a primary member of the SCY technology architecture and pedagogical agents work packages. To ensure that SAIL is of direct relevance to SCY, Slotta convened a workshop where the two lead programmers from SCY joined the three lead programmers from the TELS center and two programmers from his group at University of Toronto, to develop a new Repository Of Open source Learning Objects (ROOLO) that would interconnect with the existing SAIL portal that all three groups were using. This repository was successfully developed and is now being used in all three locations, with Slotta's team in Toronto taking a lead role.

LOOPS technology development meeting.

In conjunction with the fifth (and final) annual TELS retreat, Slotta led a break-out session of the various members of the TELS and LOOPS technology teams to discuss issues and agendas for technology development (for which Slotta has overall responsibility). Several major topics were discussed, including authoring, reporting, portals, and repositories. A SAIL technology retreat was planned where these topics would be more fully discussed, to be held in mid October, in Ontario, Canada.

Ongoing leadership and research meetings.

Professor Slotta has joined regular meetings of the LOOPS leaders where research and technology development is planned. He also convened a weekly technology development meeting, where other members of his technology group participated. Finally, Slotta and two PhD students, Cheryl Madeira and Naxin Zhao, attended bi-monthly meetings of the LOOPS research community.

University of Toronto Publications and Presentations

Krajcik, J., Slotta, J.D., McNeil, K. and Reiser, B. (2008). Designing Learning Environments to Support Students' Students Constructing Integrated Understanding. In Y. Kali, M. C. Linn, & J. E. Roseman (Eds.), Designing Coherent Science Education. New York: Teachers College Press.

Slotta, J.D. and Peters, V. (2008). A Blended Model for Knowledge Communities: Embedding Scaffolded Inquiry. Proceedings of the International Conference of the Learning Sciences. Utrecht.

Peters, V. and Slotta, J.D. (2008). Building Wiki-Based Pedagogical Scripts for Knowledge Communities. Proceedings of the International Conference of the Learning Sciences. Utrecht.

Slotta, J. D. (in press). Evolving the classrooms of the future: The interplay of pedagogy, technology and community. To appear in Mäkitalo-Siegl, K., Kaplan, F., Zottmann, J. & Fischer, F. (Eds.). Classroom of the Future. Orchestrating collaborative spaces. Rotterdam: Sense.

Slotta, J. D. & Jorde, D. (in press). Learning from our peers in international exchanges: When is worth doing, and how can we help it succeed? The International Journal of Science Education.

ii. Workshops
Slotta, J.D., Aleahmad, T., and Bannasch, S. (2008). The Scalable Architecture for Interactive Learning (SAIL) - New tools and communities for research. Workshop presentation at the International Conference of the Learning Sciences (ICLS). Utrecht, The Netherlands

Najafi, H., Slotta, J. D. & Gelb, D. (2008, August). Promoting knowledge building in graduate-level education: Opportunities and pitfalls . Paper presented at the IKIT Summer Institute, Toronto, ON.

Peters, V., & Slotta, J. D. (2008). Building wiki-based pedagogical scripts for knowledge communities. Proceedings of the International Conference of the Learning Sciences. June 23-27. Utrecht, The Netherlands.

Slotta, J.D. and Peters, V. (2008). A Blended Model for Knowledge Communities: Embedding Scaffolded Inquiry. Paper presented at the bi-annual meeting of the International Conference of the Learning Sciences (ICLS). June 23-27. Utrecht, The Netherlands.

Peters, V., & Slotta, J.D. (2008). Co-designing wiki-based scripts for secondary school biology. Paper presented at the annual meeting of the Canadian Society for Studies in Education. May 31 - June 3. Vancouver, BC.

Najafi, H., Slotta, J., Gelb, D. (2008). Understanding the relationships between curriculum planning decisions, patterns of activity, and learning outcomes: An analysis of the evolution of a course community. Paper presented at the annual meeting of the Canadian Society for the Study of Education. May 31 - June 3. Vancouver, BC.

Peters, V., & Slotta, J. D. (2008). Connecting Knowledge Building With Scripted Activities in a Secondary School Biology Classroom: A Case Study. Paper presented at the annual meeting of the American Educational Research Association, New York, NY.

University of Toronto Particpants

Jim Slotta

Jim Slotta is a co-PI of the LOOPS project and the Director of the LOOPS project at the University of Toronto.

Jim is an Associate Professor, Canada Research Chair in Education and Technology at the University of Toronto. Jim has contributed 10% of his time to the LOOPS project but from a budgetary point of view this time has been donated.

Rokham Sadeghnezhadfard

Ali Ajellu

Ali has spent 50% of his time from September 14 through September 2008 on LOOPS.

David Leung

Cheryl Madeira

Naxin Zhao

NSF Annual Progress Rpt .doc (application/msword)

LOOPS Ext Eval 2008.doc (application/msword)

LOOPSAnnualReport_UCB_final.doc (application/msword)

Toronto_Loops_annReport.doc (application/msword)

LOOPSannRpt08_kb1018.doc (application/msword)

LOOPSannRpt1021.doc (application/msword)

Papers Tinker 2007-8.doc (application/msword)

LOOPSannRpt1021v4.doc (application/msword)

LOOPSPubs_berkeley.doc (application/msword)

LOOPS Ext Eval 2008v2.doc (application/msword)